Sequence Prediction with Neural Segmental Models
نویسنده
چکیده
Segments that span contiguous parts of inputs, such as phonemes in speech, named entities in sentences, actions in videos, occur frequently in sequence prediction problems. Segmental models, a class of models that explicitly hypothesizes segments, have allowed the exploration of rich segment features for sequence prediction. However, segmental models suffer from slow decoding, hampering the use of computationally expensive features. In this thesis, we introduce discriminative segmental cascades, a multi-pass inference framework that allows us to improve accuracy by adding higher-order features and neural segmental features while maintaining efficiency. Segmental models, similarly to conventional speech recognizers, are typically trained in multiple stages. In the first stage, a frame classifier is trained, and in the second stage, segmental models are trained with the outputs of the frame classifier. Both training stages require manual alignments, and obtaining manual alignments are time-consuming and expensive. We explore end-to-end training for segmental models with various loss functions, and show how end-to-end training with marginal log loss can eliminate the need for detailed manual alignments. We draw the connections between the marginal log loss and a popular end-to-end training approach called connectionist temporal classification, and present a unifying framework for various end-to-end graph search-based models, such as hidden Markov models, connectionist temporal classification, and segmental models. Finally, we discuss possible extensions of segmental models to large-vocabulary sequence prediction tasks. Thesis Supervisor: Karen Livescu Title: Associate Professor
منابع مشابه
Prediction of Gain in LD-CELP Using Hybrid Genetic/PSO-Neural Models
In this paper, the gain in LD-CELP speech coding algorithm is predicted using three neural models, that are equipped by genetic and particle swarm optimization (PSO) algorithms to optimize the structure and parameters of neural networks. Elman, multi-layer perceptron (MLP) and fuzzy ARTMAP are the candidate neural models. The optimized number of nodes in the first and second hidden layers of El...
متن کاملPrediction of Gain in LD-CELP Using Hybrid Genetic/PSO-Neural Models
In this paper, the gain in LD-CELP speech coding algorithm is predicted using three neural models, that are equipped by genetic and particle swarm optimization (PSO) algorithms to optimize the structure and parameters of neural networks. Elman, multi-layer perceptron (MLP) and fuzzy ARTMAP are the candidate neural models. The optimized number of nodes in the first and second hidden layers of El...
متن کاملPrediction of structural forces of segmental tunnel lining using FEM based artificial neural network
To judge about the performance of designed support system for tunnels, structural forces i.e. peak values of axial and shear forces and moments are critical parameters. So in this study, at first a complete database using finite element method was prepared. Then, a model of artificial neural network (ANN) using multi-layer perceptron was developed to estimate lining structural forces. Sensitivi...
متن کاملComparison of Artificial Neural Network and Regression Models for Prediction of Body Weight in Raini Cashmere Goat
The artificial neural networks (ANN) are the learning algorithms and mathematical models, which mimic the information processing ability of human brain and can be used to non linear and complex data. The aim of this study was to compare artificial neural network and regression models for prediction of body weight in Raini Cashmere goat. The data of 1389 goats for body weight, height at withers ...
متن کاملComparison of Artificial Neural Networks and Cox Regression Models in Prediction of Kidney Transplant Survival
Cox regression model serves as a statistical method for analyzing the survival data, which requires some options such as hazard proportionality. In recent decades, artificial neural network model has been increasingly applied to predict survival data. This research was conducted to compare Cox regression and artificial neural network models in prediction of kidney transplant survival. The prese...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1709.01572 شماره
صفحات -
تاریخ انتشار 2017